NASM version 2.14.02 (from Debian testing) also does not get any errors here:

source$ /usr/bin/nasm -v
NASM version 2.14.02
source$ /usr/bin/nasm debug.asm -I ../../lmacros/
source$

Updated to the actual most recent commit, nasm-2.14.03rc2-369-ga2c1c7d0, same kinds of errors:

source$ nasm -v
NASM version 2.15rc0 compiled on Aug 10 2019
source$ nasm debug.asm -I ../../lmacros/
debugtbl.inc:72: error: control character in string not allowed here
debugtbl.inc:74: warning: unterminated string [-w+other]
debugtbl.inc:74: error: expression syntax error
debugtbl.inc:75: error: expression syntax error
asmtabs.asm:366: error: symbol `OPLIST_01' not defined
asmtabs.asm:366: error: symbol `OPLIST_02' not defined
asmtabs.asm:366: error: symbol `OPLIST_3D' not defined
asmtabs.asm:366: error: symbol `OPLIST_3E' not defined
asmtabs.asm:366: error: symbol `OPLIST_3F' not defined
asmtabs.asm:366: error: symbol `OPLIST_40' not defined
asmtabs.asm:366: error: symbol `OPLIST_41' not defined
source$ nasm debug.asm -I ../../lmacros/
debugtbl.inc:12: error: control character in string not allowed here
debugtbl.inc:71: error: control character in string not allowed here
debugtbl.inc:73: error: expression syntax error
debugtbl.inc:74: warning: unterminated string [-w+other]
debugtbl.inc:74: error: expression syntax error
asmtabs.asm:366: error: symbol `OPLIST_01' not defined
asmtabs.asm:366: error: symbol `OPLIST_02' not defined
asmtabs.asm:366: error: symbol `OPLIST_3D' not defined
asmtabs.asm:366: error: symbol `OPLIST_3E' not defined
asmtabs.asm:366: error: symbol `OPLIST_3F' not defined
asmtabs.asm:366: error: symbol `OPLIST_40' not defined
asmtabs.asm:366: error: symbol `OPLIST_41' not defined
source$

Still occurs in https://repo.or.cz/nasm.git/commitdiff/7eb18213b78f06b45c85e2b224613cce4a20304b

By the way, I can reproduce seemingly random errors on my partner's server (Debian 9). Here, the errors are always the same when the input stays the same. However, modifying the source ever so slightly without any changes that should affect the errors, does cause a different set of errors.

If need be (if you cannot reproduce these errors else), I can probably set up an ssh account for that server.

Still occurs in https://repo.or.cz/nasm.git/commitdiff/dd88aa9a1b065bb28eb4535a4382c3fe36c5fa31 -- though with different errors than the prior test.

This MIGHT be fixed with some preprocessor fixes I just checked in, but I haven't yet had a time to test out this test case myself. However, the symptoms appear very similar.

Both the assembly table testcase and another new testcase still fail in a similar way. Again the precise errors changed from the prior commit to this one, but they're of the same kinds.

Building this script is the second testcase, newnasm is https://repo.or.cz/nasm.git/commitdiff/f7dbdb2e136db99051b14403a0f29c5155bbf7d8 here (used only for building bootimg.asm):

https://hg.ulukai.org/ecm/bootimg/rev/32faed02b0db

bootimg$ (cd ../ldosboot; oldnasm boot.asm -I ../lmacros/ -D_LOAD_NAME="'LDEBUG'" -D_FAT16=0 -D_MAP=boot12.map -l boot12.lst -o boot12.bin) && newnasm bootimg.asm -I ../lmacros/ -I ../ldosboot/ -o bootimg.img -l bootimg.lst -D_PAYLOADFILE="'../ldebug/bin/ldebug.com',::empty,::chdir,'ddebug','../ldebug/bin/lddebug.com',::chdir,'subdir','bootimg.asm',::chdir,..,::chdir,.." -D_BOOTFILE="'boot12.bin'" && dosemu -I "floppy { device bootimg.img threeinch }" -A -dumb
boot.asm:1328: warning: FAT12: 7 bytes still available. [-w+user]
bootimg.asm:627: error: Too long name part in '../ldebug/bin/ldebug.com'
bootimg.asm:627: error: _fill: data too large, exceeds size
bootimg.asm:627: error: TIMES value -7 is negative
bootimg.asm:627: error: TIMES value -7 is negative
bootimg.asm:627: error: Too long name part in 'ddebug'
bootimg.asm:627: error: _fill: data too large, exceeds size
bootimg.asm:627: error: TIMES value -6 is negative
bootimg.asm:627: error: TIMES value -6 is negative
bootimg.asm:627: error: TIMES value -6 is negative
bootimg.asm:627: error: Too long name part in '../ldebug/bin/lddebug.com'
bootimg.asm:627: error: _fill: data too large, exceeds size
bootimg.asm:627: error: TIMES value -7 is negative
bootimg.asm:627: error: TIMES value -7 is negative
bootimg.asm:627: error: Too long name part in 'subdir'
bootimg.asm:627: error: _fill: data too large, exceeds size
bootimg.asm:627: error: TIMES value -7 is negative
bootimg.asm:627: error: TIMES value -7 is negative
bootimg.asm:627: error: TIMES value -7 is negative
bootimg.asm:627: error: Too long name part in 'bootimg.asm'
bootimg.asm:627: error: _fill: data too large, exceeds size
bootimg.asm:627: error: TIMES value -7 is negative
bootimg.asm:627: error: TIMES value -7 is negative
bootimg$

Same file with same options but an older NASM succeeds without errors or warnings. You can provide arbitrary filenames instead of the ones I used, it probably doesn't matter. For example, this fails too:

bootimg$ (cd ../ldosboot; oldnasm boot.asm -I ../lmacros/ -D_LOAD_NAME="'LDEBUG'" -D_FAT16=0 -D_MAP=boot12.map -l boot12.lst -o boot12.bin) && newnasm bootimg.asm -I ../lmacros/ -I ../ldosboot/ -o bootimg.img -l bootimg.lst -D_PAYLOADFILE="'boot12.bin',::empty,::chdir,'subdir','bootimg.asm',::chdir,.." -D_BOOTFILE="'boot12.bin'"
boot.asm:1328: warning: FAT12: 7 bytes still available. [-w+user]
bootimg.asm:627: error: Too long name part in 'boot12.bin'
bootimg.asm:627: error: _fill: data too large, exceeds size
bootimg.asm:627: error: TIMES value -7 is negative
bootimg.asm:627: error: TIMES value -7 is negative
bootimg.asm:627: error: Too long name part in 'subdir'
bootimg.asm:627: error: _fill: data too large, exceeds size
bootimg.asm:627: error: TIMES value -7 is negative
bootimg.asm:627: error: TIMES value -7 is negative
bootimg.asm:627: error: TIMES value -7 is negative
bootimg.asm:627: error: Too long name part in 'bootimg.asm'
bootimg.asm:627: error: _fill: data too large, exceeds size
bootimg.asm:627: error: TIMES value -7 is negative
bootimg.asm:627: error: TIMES value -7 is negative
bootimg$

Still occurs with https://repo.or.cz/nasm.git/commitdiff/9781db8906a769507a85280829c42dfe7b02f600

source$ newnasm -v
NASM version 2.15rc0 compiled on Sep 28 2019
source$ newnasm debug.asm -I ../../../lmacros/ 2>&1 | head
debugtbl.inc:59: error: expression syntax error
debugtbl.inc:62: error: control character in string not allowed here
debugtbl.inc:79: error: expression syntax error
debugtbl.inc:85: error: control character in string not allowed here
debugtbl.inc:86: error: control character in string not allowed here
debugtbl.inc:86: error: `%assign' expects a macro identifier
debugtbl.inc:90: error: expression syntax error
debugtbl.inc:130: error: expression syntax error
debugtbl.inc:133: error: expression syntax error
debugtbl.inc:136: error: expression syntax error
source$ 


This is nondeterministic even on the server without modifying the source:

source$ newnasm debug.asm -I ../../../lmacros/ 2>&1 | head
debugtbl.inc:59: error: expression syntax error
debugtbl.inc:62: error: control character in string not allowed here
debugtbl.inc:62: error: expression syntax error
debugtbl.inc:74: warning: unterminated string [-w+other]
debugtbl.inc:74: warning: trailing garbage after expression ignored [-w+other]
debugtbl.inc:83: error: expression syntax error
debugtbl.inc:89: error: control character in string not allowed here
debugtbl.inc:130: error: expression syntax error
debugtbl.inc:133: error: expression syntax error
debugtbl.inc:136: error: expression syntax error
source$ newnasm debug.asm -I ../../../lmacros/ 2>&1 | head
debugtbl.inc:59: error: expression syntax error
debugtbl.inc:74: warning: unterminated string [-w+other]
debugtbl.inc:74: warning: trailing garbage after expression ignored [-w+other]
debugtbl.inc:77: error: expression syntax error
debugtbl.inc:82: warning: unterminated string [-w+other]
debugtbl.inc:82: warning: trailing garbage after expression ignored [-w+other]
debugtbl.inc:90: error: expression syntax error
debugtbl.inc:130: error: expression syntax error
debugtbl.inc:133: error: expression syntax error
debugtbl.inc:136: error: expression syntax error
source$ newnasm debug.asm -I ../../../lmacros/ 2>&1 | head
debugtbl.inc:59: error: expression syntax error
debugtbl.inc:62: error: expression syntax error
debugtbl.inc:64: error: control character in string not allowed here
debugtbl.inc:74: warning: unterminated string [-w+other]
debugtbl.inc:74: warning: trailing garbage after expression ignored [-w+other]
debugtbl.inc:83: error: expression syntax error
debugtbl.inc:88: error: control character in string not allowed here
debugtbl.inc:130: error: expression syntax error
debugtbl.inc:133: error: expression syntax error
debugtbl.inc:136: error: expression syntax error
source$

Another error occurs seldom now:

source$ newnasm debug.asm -I ../../../lmacros/ 2>&1 | head
debugtbl.inc:59: error: expression syntax error
debugtbl.inc:62: error: expression syntax error
debugtbl.inc:64: error: control character in string not allowed here
debugtbl.inc:74: warning: unterminated string [-w+other]
debugtbl.inc:74: warning: trailing garbage after expression ignored [-w+other]
debugtbl.inc:83: error: expression syntax error
debugtbl.inc:88: error: symbol `�UE' not defined before use
debugtbl.inc:89: error: symbol `iق�UF' not defined before use
debugtbl.inc:90: error: control character in string not allowed here
debugtbl.inc:130: error: expression syntax error
source$

Created attachment 411743 [details]
lDebug assembler tables test case with nondeterministic errors

Here's the test case in a .tar.xz file, with files from https://hg.ulukai.org/ecm/ldebug and https://hg.ulukai.org/ecm/lmacros

The bootimg test seems like it may suffer from the bug described in https://bugzilla.nasm.us/show_bug.cgi?id=3392621#c1

The other error may be in %strcat handling, in the following hunk:

@@ -3491,21 +3552,21 @@ issue_error:
          * zero, and a numeric token to use as an expansion. Create
          * and store an SMacro.
          */
-        macro_start = new_Token(NULL, TOK_STRING, NULL, 0);
-        macro_start->text = nasm_quote(pp, len);
+        macro_start = make_tok_qstr(pp);
         nasm_free(pp);
         define_smacro(ctx, mname, casesense, 0, macro_start);
         free_tlist(tline);
         free_tlist(origline);
         return DIRECTIVE_FOUND;
 

From https://repo.or.cz/nasm.git/commitdiff/8b262474424c0f6912b22bbf7452f26bfa4d1235

In https://repo.or.cz/nasm.git/blame_incremental/d03a6c8ffedd1496221eca5f02ba0215794cfaf9:/asm/preproc.c#l4199 I think we can see that the string is not zero-terminated. However, make_tok_qstr expects a zero-terminated string as it uses strlen to determine the length. Prior to the hunk I quoted, nasm_quote was passed the length in its second parameter.

This patch fixes both test cases for me. My understanding of the two error sites seems to be correct!


diff --git a/asm/preproc.c b/asm/preproc.c
index 14a73885..647e537c 100644
--- a/asm/preproc.c
+++ b/asm/preproc.c
@@ -612,6 +612,7 @@ static Token *expand_smacro(Token * tline);
 static Token *expand_id(Token * tline);
 static Context *get_ctx(const char *name, const char **namep);
 static Token *make_tok_num(Token *next, int64_t val);
+static Token *make_tok_qstr_with_len(Token *next, const char *str, size_t len);
 static Token *make_tok_qstr(Token *next, const char *str);
 static Token *make_tok_char(Token *next, char op);
 static Token *new_Token(Token * next, enum pp_token_type type,
@@ -4236,7 +4237,7 @@ issue_error:
          * zero, and a numeric token to use as an expansion. Create
          * and store an SMacro.
          */
-        macro_start = make_tok_qstr(NULL, qbuf);
+        macro_start = make_tok_qstr_with_len(NULL, qbuf, len);
         nasm_free(qbuf);
         define_smacro(mname, casesense, macro_start, NULL);
         free_tlist(tline);
@@ -4313,7 +4314,7 @@ issue_error:
 
 	txt = (start < 0) ? "" : tok_text(t) + start;
 	len = count;
-        macro_start = make_tok_qstr(NULL, txt);
+        macro_start = make_tok_qstr_with_len(NULL, txt, len);
 
         /*
          * We now have a macro name, an implicit parameter count of
@@ -6563,13 +6564,18 @@ static Token *make_tok_num(Token *next, int64_t val)
 }
 
 /* Create a quoted string token */
-static Token *make_tok_qstr(Token *next, const char *str)
+static Token *make_tok_qstr_with_len(Token *next, const char *str, size_t len)
 {
-    size_t len = strlen(str);
     char *p = nasm_quote(str, &len);
     return new_Token_free(next, TOK_STRING, p, len);
 }
 
+static Token *make_tok_qstr(Token *next, const char *str)
+{
+    size_t len = strlen(str);
+    return make_tok_qstr_with_len(next, str, len);
+}
+
 /* Create a single-character operator token */
 static Token *make_tok_char(Token *next, char op)
 {

I just checked in a modified version of your patch. *Huge thanks* for your help tracking this down. If you could confirm that it works for you, that would be great.

master checkin 18f413422221063d524f7a5b08bece5f8604af27

*** Bug 3392621 has been marked as a duplicate of this bug. ***

I think the "*q = '\0';" line is wrong, because the allocated buffer doesn't have space for a terminator. It is also not needed for make_tok_qstr_len.

Everything with %strcat and %substr works as intended now, as of https://repo.or.cz/nasm.git/commitdiff/f770ce8be48f17b772c3a9c32b3da944e155f502